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Abstract 

A relationship between the Fisher information and the characteristic function is estabhshed with the help of 
two inequalities. A necessary and sufficient condition for equality is found. These results are used to determine the 
asymptotic efficiency of a distributed estimation algorithm that uses constant modulus transmissions over Gaussian 
multiple access channels. The loss in efficiency of the distributed estimation scheme relative to the centralized 
approach is quantified for different sensing noise distributions. It is shown that the distributed estimator does not 
^ ' incur an efficiency loss if and only if the sensing noise distribution is Gaussian. 
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i£j' I. Introduction 

We investigate the relationship between the Fisher information about a location parameter and the 

>• characteristic function of the additive noise by providing a new derivation for two inequalities that involve 

^ the Fisher information and the characteristic function. These inequalities were originally derived using 

Tjj- a different approach and applied in a quantum physics setting to estimate the survival probability of a 

^ — J quantum state in [1]. Conditions for equality are also delineated herein for the first time in the literature, 

t^^ and used to investigate the asymptotic efficiency of a distributed estimation scheme over a Gaussian 

S multiple- access channel. 



II. The Inequalities 

S^ . Consider a model where a deterministic location parameter, 9, is related to observations xi = + t]i, 
H ' I = 1, . . . , L, where r]i are iid and real- valued random variables. Let the characteristic function of r]i be 
■ - - ^{oo) ■.= E[e^^'^'] and let the Fisher information be defined as [l2l|, jSl 

nv) ■■= r ^^^^ < ^' (1) 

where p{x) is the pdf of r]i, assumed to be continuously differentiable, and with support (— oo, oo). Note 
that I{r]) is the Fisher information in xi about 9, and is a deterministic value which does not depend on 
9. In the following, i] denotes a random variable with the same distribution as any r]i. 

We present the following theorem, which provides two bounds involving /(r/) and ^(uj). It was proved 
first in [IJ using the Cramer- Rao inequality. We provide an alternate proof which also delineates the 
condition for equality for the first time in the literature. The condition for equality will be central in 
Section Hn] to establish necessary and sufficient conditions for the asymptotic efficiency of a distributed 
estimation algorithm over a Gaussian multiple-access channel. 
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with the SenSIP Center, School of Electrical, Computer and Energy Engineering, Fulton Schools of Engineering, Arizona State University, 
Tempe, AZ 85287. USA. (Email: {cihan, maheshkb, spanias} @asu.edu). 
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Fig. 1. System model: Wireless sensor network. The estimator is located at the fusion center. 
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Theorem 1: Let ipR{uj) and (^/(u;) be the real and the imaginary parts of (^{co), respectively. We have 

'1 

2 

'1 

^2 

with equality in both © and Q if and only if w = 0. 

Proof: Let s{x) := p' (x) / p{x) be the score function, where we recall that p(x) is the pdf of rji. Let 
g{x) be a differentiable function satisfying liia^^^^ g{x)p{x) = 0. Using Stein's identity flU Lemma 
1.18], we have 

E[g{r])s{r])] = -E[g'{7^)]. (4) 

Applying the Cauchy-Schwarz inequality yields 



E'lg'in)] < Iiv)E[g\v)], 



(5) 



with equality if and only if s(x) = ag{x) for some a and all x. By substituting gi{x) := cos(wx) — (^r{uj) 
for g{x) in ([5]), equation ^ is obtained. Similarly, 5'2(a^) :=sin(u;x) — ^i{uj) substituted for g(x) yields 
equation ^. 

To examine when equality occurs, first note that if a; = 0, since ^r{0) = 1 and v3/(0) = 0, equations ^ 
and ^ become equalities. Conversely, consider cu ^ 0. The equality condition for ([3]) is s{x) = ag2{x), 
which yields the first order differential equation 



p'{x) 
p{x) 



a sm cjx 



(fiiiu)] 



(6) 



which must provide a solution satisfying p{x) > and f^ p{x)dx = 1. The solution to © is of the 
form p{x) = C'e^°^''^^*^'^^e^S'=°''('^^), which is unbounded as x — > — oo when (fiico) ^ 0, and periodic when 
^i{uj) = 0. In either case, J^ p{x)dx = 1 is not possible. This shows that there is no pdf satisfying Q 
when CO ^ 0, and therefore, equality in ^ cannot be attained for tu ^ 0. The same conclusion can be 
drawn about equation ^, using a similar argument with s(x) = agi{x). ■ 



in. Application to Distributed Estimation 

A sensor network, illustrated in Figure [H consisting of L sensors is considered. The value, xi, observed 
at the Z*'^ sensor is 

e + VI (7) 



Xl 



for / 



L, where 6* is a deterministic, real-valued, unknown parameter in a bounded interval of 



known length, [0,^/?], where 6r < oo, and rji are iid real-valued random variables. We will assume that 



rii has zero mean and variance o"^, when the mean and variance exist. Due to constraints in the transmit 
power, we consider a scheme where the l^'^ sensor transmits its measurement, xi, using a constant modulus 
base-band equivalent signal, ^Jpe^^^', over a Gaussian multiple access channel so that the received signal 
at the fusion center is given by 



Vl 



^^e-'-. + z/, (8) 



1=1 



where the transmitted signal at each sensor has per-sensor power of p,uj E (0, 2n/9f(] is a design parameter 
to be optimized, and u ~ CJ\f{0, al) is independent of {r]i}jl^j^. Note that the restriction to E (0, 2n/6j^] is 
necessary even in the absence of sensing and channel noise (yL = y/pe^"^^) to uniquely determine 9 from 

Vl. 

In a centralized problem, 9 is estimated from {x/}^^. The Cramer- Rao bound is the well known 
benchmark on the variance of unbiased estimators with finite samples and is proportional to [/(r/)]^^ 
|[5l pp. 120]. For large L, the asymptotic variance is an appropriate performance metric. Under certain 
regularity conditions, the benchmark on the asymptotic variance is given by [I{r])]~^ [|5l pp. 439]. Hence, 
the Fisher information has a central role to play in establishing benchmarks for the estimation of a location 
parameter for centralized estimation problems which address estimators of 9 based on {xi}JL-^^. 

For the distributed setting, based on ([8]), the estimators of 9 rely on y^. The desire to have constant 
modulus transmissions over a Gaussian multiple-access channel causes the fusion center in Figure \T\ to 
have access to only yL, rather than {xi}JL^. Clearly, yL has less information about 9 than {xi}fLi. In what 
follows, we quantify this loss by examining the efficiency of the minimum (asymptotic) variance estimator, 
and comparing it with the benchmark for the centralized problem, [/(r/)]^^, for different distributions on 
the sensing noise, rj. Using Theorem [H it is shown that there is no loss in efficiency if and only if r] is 
Gaussian. 

A. The Estimator 

To estimate 9, we normalize yL in ^ and define: 



zl 



■.= ^i = V~Pe^-'ji2''^" + T^ (9) 



L^^ L' 

1=1 



where zl = \zl\ exp{jZzL) = z^+jzl, and zf and zl are the real and imaginary parts of zl, respectively. 
Also zl: = [zl z^Y and 7.{9) ■.=[E[zl\ E[zl]\^ = ^[^p li{uo) cos uj9 — ipi{(jj)sm.uj9 (pFi{u)smu9 + 
fi{u) sina;6']"^. 

Given yL (or equivalently zl), the estimator with the smallest asymptotic variance is given by ^ (3.6.2), 
pp. 82] 

9l = argmin[zi - z{9)]1:-\9)[zl - z{9)f, (10) 



where 

no) 



Sll(^) Si2(^) 

^2l{9) S22W 



(11) 



is the 2 X 2 asymptotic covariance matrix of zl, satisfying lim^^oo V^[zl — z(6')] = A/'(0, E(6')). Its 
elements are given by 

Sii(6') = p [vc cos'^{uj9) + Vs sin^(a;^)] 
S22(^) = P \vsCos'^{uj9) + t>csin^(a;^)] 
Ei2(6') = T,2i{9) = p{vc - Vs) sin(a;6') cos(a;6'), 

where fc :=var[cos((X'?7/)] = 1/2 + (pfj^{2u)/ 2 — (p'j^{uj) and f^ := var[sin(u;r7j)] = 1/2 — (pLj^{2u) /2 ~ (p'j{u). 



Estimators of the form in (flOl ) have an asymptotic variance given by |l6l Lemma 3.1] 



AsV(a;) 



dz{9) 
39 



^-\o) 



dz{9) 
09 



-, -1 



(12) 



Substituting dz{9)/d9 = y/puj[—ipR{u) smu9 — (pi{u) cosu9 (Pr{ui) cosu9 — (pi{u) smuj9]'^ and H ^{9) 
whose elements can be expressed in terms of Sii(6'), T.22i9) and ^12(6'), the asymptotic variance is given 
by 



AsV(a;) 



2VcVs 



(1 + ipni2u) - 2ipliuj)) (1 - ipni2uj) - 2ipjiu)) 
u^ [ipliu) (1 + ipni2cu) - 2(^1(0;)) + ipj{u;) (1 - ipR{2co) - 2ipj{co))] 



(13) 



Note that AsV(u;) depends on the sensing noise through its characteristic function, and does not depend 
on the channel noise variance, cr^, which washes out for large L. 



B. Asymptotic Efficiency 

We now address the asymptotic efficiency of 9^ and characterize the condition under which AsV(cj) 
can be made arbitrarily close to [I(r])]^^: 

Theorem 2: The estimator in (fTOl) can be arbitrarily close to being asymptotically efficient by the proper 
choice of u, that is, 

inf AsV(a;) = -l^, (14) 

uje(o,2n/eii] I{r]) 

if and only if r] is Gaussian. 

Proof: We begin by showing that if (fT4l) holds, then r] is Gaussian. Using Theorem [H the inequalities 
in ^ and ([3]) can be rewritten for w > as 



wVK^) 



(1 [1 + Mu^)] - ^IM) 



< Hv), 



(15) 
(16) 



(i [1 - M^)] - ^li^)) 

where we use that when a; 7^ 0, Q and ([3]) are strict inequalities. Adding the inequalities in (fT5l) and 
(fT6l) . rearranging the resulting inequality and recalling (fT3l) . we have 

1 



liv) 



< AsV(a;) 



uei0,2n/9n]. 



(17) 



Equation ([17]) indicates that the infimum in (fT4)) is not attained for any non-zero finite value of tu. Since tu 
is bounded above, the only way for (fT4l) to hold is when lim^^o AsV(cj) = [I{r])]~^. It is easy to verify, 
using L'Hospital's rule, that lim^j^o AsV(a;) = a^, the variance of t]i. Therefore, for (fT4l) to hold, we have 
[/(?7)]^^ = cr^. The only distribution that satisfies this is the Gaussian dH Lemma 1.19]. This completes 
the proof of the first half. 

To show that (fT4l) holds when rji is Gaussian, (p{uj) = e~^ "^1"^ is substituted into (fT3l) to yield: 



AsV(a; 
which is non-decreasing in w, since 

9AsV(u;) 2e 



UJ^ 



,e--¥' [^-¥' - 1 



-2(ji^uj'- 



dbj 



U'' 



g2<72<^2 



2, ,2 



,2, .2„2aiui^ 



1 - e^'""'^ ) + 2a^u^ + 2af^cu'e'''^ 



>0, 



(18) 



(19) 



for a; > 0. ■ 

The phase modulated scheme considered here has the advantage of constant modulus transmissions. 
Due to the use of phase modulation, the result in Theorem [21 is related to the efficiency of the estimator of 
a location parameter using the empirical characteristic function (ECF), defined as (p{uj) := L^^ J2i=i e^'^^K 
It can be seen from ^ that zl = y/pe^'^^(f>{uj) + u/L is related to the ECF through scaling and additive 
noise. The efficiency of empirical characteristic function based estimators has been considered for arbitrary 
parameters (that is, not just location parameters) in [T|, but with a continuum of infinitely many values 
of the argument, u, of the ECF. In the current distributed estimation application, the evaluation of (p{uj) 
for many values of oj at the fusion center corresponds to many transmissions per sensor observation, 
requiring large bandwidth. In contrast, we consider a single value of tu for estimation, requiring a single 
transmission per sensor. The analog transmissions are assumed to be appropriately pulse-shaped and phase 
modulated to consume finite bandwidth. 

When the sensing noise distribution is symmetric, the cost function on the right hand side of (fTOl) that 
needs to be minimized can be expressed as 



c{9)=[zL-m]^'\o)[zL-mf 

=—^ - 4:p^^^Vs<^{co)[zism{ue) + z^cos{uje)] + 2p\,(/?2(w) 

Ap VcVg - 

+ p{vc - Vs) {{ziY - (z^f) cos{2uje) - 2p{v, + Vs)ziz^ sm{2uje) 

+ p{v^ + vs) {{zir + (z^r) 



(20) 



Differentiating with respect to 9, we have 



dciO) 

de 



2uzi; cos{ui 

PVcVs 



Zj , , 

— ^ — tan((X't 



X 



1 + ^ tan(a;6') h;^ + U - 



v^<^(w) 



The values of Q at which (1211) is zero are given by 

, nvr ± ^ 1 

Q e < '-, -ZzL, 



zf; cos{u9) z 
Zzl + 2n7r ± f 



tan(a;6') v 



u 



(21) 



(22) 



where cu ^ and n G Z+. The value of 9 that minimizes c{9) is easily verified by substituting the values 
of 9 from (|22)) into (|20l) and is given by 



9 



u 



-^zl. 



(23) 



Hence, in the presence of symmetric noise, the estimator in (|TOl) that minimizes the asymptotic variance 
reduces to the simple expression in (l23l) . which was first considered in [8J. However, in [8J, neither the 
optimality (in terms of minimizing the asymptotic variance) nor the efficiency of the estimator in (l23l) 
was considered. 



C. Quantifying Relative Efficiency 

One way of interpreting Theorem [2] is to observe that when the sensing noise is Gaussian, no information 
is lost by analog phase modulation if to is chosen sufficiently small. On the other hand, information is 
lost when the sensing noise follows other distributions. To see this more clearly, we define the relative 
efficiency between the asymptotic variance and the Fisher information as: 



£iv) 



liri) inf AsVrw) 



(24) 



Distribution 


Gaussian 


Laplace 


Cauciiy 


Uniform 


£iv) 


1 


2/3 


0.5c-'e-''{l-e-'')-' « 0.65 






TABLE I 

£{rj) FOR DIFFERENT DISTRIBUTIONS. 



It can easily be verified that S{r]) is scale-invariant in the sense that £{ar]) = £{r]) for any a G M. 
Moreover, based on Theorem |2] and (flTl) . < £{t]) < 1, where the equality in the upper-bound is 
achieved only if r] is Gaussian. 

The relative efficiency in (|24l) depends only on the distribution of the sensing noise. The values of S{r]) 
for several distributions are provided in Table HI The result in Table IJ for the Gaussian case has been 
established in Theorem [2l For the Laplace sensing noise, (^(co) = (1 + c<;^cr^/2)~^, AsV(u;) = (T^(1 + 
(T^w^/2)/(l + 2cr^uj^), and inf^g(o,27r/0fl] AsV(a;) = 3(t,^/4, by inspecting the third derivative of AsV(w). 
Similarly for the Cauchy case, ip{uj) = e'^^, AsV(w) = e^^'^{l-e-^^'^)/2uj^, and inf^e(o,2^/e^] AsV(w) = 
47^e'^(l — e~'^^)/c^, by examining the first derivative of AsY(uj) where 7 is the scale parameter of the 
Cauchy random variable, c:=2 + W{—2e^'^), and W{-) is the Lambert W^-function [9]. For the uniform 
distribution, an extension of the definition in ([T]) can be used to argue that the Fisher information is infinite 
llli pp. 119], and the relative efficiency of the estimator as defined in (|24|) is zero. 

We have seen that the Gaussian sensing noise is the only distribution with the highest possible efficiency 
when the observations xi are transmitted with phase modulation over Gaussian multiple-access channels 
and the estimator in (flOl) is used. However, it is possible that other sensing noise distributions, which 
yield less efficiency, have better asymptotic variances. This is because efficiency is defined relative to the 
Fisher information. For example, for Laplace sensing noise, the proposed estimator is not asymptotically 
efficient, but has better asymptotic variance than in the Gaussian case, since its inverse Fisher information, 
[I{t])]^^, is lower. In conclusion, Gaussian sensing noise has the only distribution that does not suffer a 
loss in efficiency when the sensed data xi is mapped to constant modulus transmissions over Gaussian 
multiple-access channels. 

IV. Numerical Results 

In Figures [2] and |3l the asymptotic variance and the value of [I{r])]~^ in dB are plotted versus cu, when 
the sensing noise is Gaussian, Laplace, uniform and Cauchy distributed. 

From Figure |2l it can be seen that the asymptotic variance approaches [I(r])]^^ only as tj — )• for 
Gaussian sensing noise, and is bounded away from [/(?7)]~^ for other values of tu. The estimator in (flOl ) 
is not efficient when the sensing noise is non-Gaussian. Using the definition of relative efficiency in (|24l) . 
it can seen from Figure |2] that S{t]) in the case of Gaussian sensing noise is OdB, and in the case of 
Laplace sensing noise is about — 3.5dB. In Figure [21 it can be verified that inf^^ AsV(u;) ~ 0.75, which is 
about — 2.5dB at u; = 1/v^, which is lower than the Gaussian sensing noise case. 

From Figure [3] the relative efficiency for the Cauchy noise case is about — 3.8dB, verifying the value 
shown in Table HI The inverse Fisher information for the uniform case is (—00 dB) and is not shown in 
Figure [3l The relative efficiency as defined in (|24|) . for uniform noise, is therefore zero. When the sensing 
noise follows the Cauchy, uniform or Laplace distributions, the estimator is not asymptotically efficient. 

V. Conclusions 

In this paper, we considered the relationship between the Fisher information and the characteristic 
function through two bounds. The condition for equality was also derived, for the first time in literature. 

This result was used to prove the asymptotic efficiency of a distributed estimator that minimizes the 
asymptotic variance in the presence of Gaussian sensing noise. In all cases, the loss in efficiency was 
quantified through a scale-invariant relative efficiency metric that takes values between and 1. This 
metric depends only on the distribution of the sensing noise used, and was computed for the Gaussian, 
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Fig. 2. Plot of asymptotic variance vs. w. 



Laplace, Cauchy and uniform cases. These relative efficiency values can be interpreted as the amount of 
information lost due to constant modulus transmissions over Gaussian multiple-access channels relative 
to having perfect access to all sensor measurements. Numerical evaluations confirm the result that the 
estimator in (flOl) is asymptotically efficient only when the sensing noise is Gaussian. 
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Fig. 3. Plot of asymptotic variance vs. (jJ. Note that the value of [/(?;)] ^ is (— oo dB) for the uniform sensing noise case and is not 
shown. 



